
## Portable WebAssembly

One of the most common misconceptions about WebAssembly comes from the name itself.
Although WebAssembly was originally designed for use in web browsers, it is not tied to any Web
APIs at all, and is in fact a highly portable binary format that can also be run in
[non-web](https://webassembly.org/docs/non-web/) environments.

<figure>
  <img src="/blog/2021/09/wasm-ds.jpg" />
</figure>

Because of WebAssembly's very simple instruction set, it is possible to build runtimes for a wide range of target architectures
and host environments. Wasm3, a WebAssembly interpreter with minimal dependencies, has been shown to
be easy to compile for other embedded systems [such as the Arduino](https://github.com/wasm3/wasm3-arduino), which can then
dynamically load and execute WASM modules.

<Github
  author="wasm3"
  repo="Wasm3"
  description="🚀 The fastest WebAssembly interpreter, and the most universal runtime"
/>

As I'd recently been experimenting with WebAssembly in non-browser environments,
running WebAssembly on the Nintendo DS (one of my favourite portable devices) seemed like a good candidate for another experiment.

## Homebrew Software

I've long been interested in homebrew applications and games, as well as the community of enthusiastic developers building
applications for popular devices.

Using the excellent tools developed by devkitPro, in particular devkitARM and [libnds](https://libnds.devkitpro.org/), it's possible to develop programs
than can utilise most features of the Nintendo DS and run them on real devices.

An example of a very simple homebrew Nintendo DS game is Snake-DS by PolyMars, which in turn utilises
[NFLib](https://sourceforge.net/projects/nflib/) on top of libnds to abstract away some of the unusual quirks of the DS's architecture.

<Github
  author="PolyMarsDev"
  repo="Snake-DS"
  description="A simple snake clone for the Nintendo DS"
/>

For an excellent guide on Nintendo DS development in general, [see here](https://patater.com/files/projects/manual/manual.html).

Due to the Snake-DS's low complexity, it seemed like an ideal candidate
to rewrite in [AssemblyScript](https://www.assemblyscript.org/), a special variant of TypeScript that compiles directly to WebAssembly.

## Building Wasm3 for Nintendo DS

Unlike many other WASM runtimes (such as [wasmer](https://wasmer.io/)), Wasm3 is purely a WebAssembly interpreter without any AOT required nor
JIT included.

While this can result in less than optimal performance at run time, it allows for much more flexibility in terms of portability. WASM modules can be dynamically loaded and swapped out
at run time, and the memory profile is a lot more predictable than including a JIT - which is particularly important for
systems with strict memory constraints such as the Nintendo DS.

Using the devkitARM port of GCC, it is simple to build Wasm3 as a library for use in a libnds project.
A full build pipeline and compiled library for `wasm3-nds` is available here:

<Github
  author="moxon6"
  repo="nds-dependencies"
  description="🎮 Repo for building dependencies for NDS apps"
/>

## Communicating between C++ and WebAssembly

<figure>
  <img src="/blog/2021/09/as-cpp.svg" />
</figure>

Loading a WASM module with Wasm3 is relatively easy. This project opted to follow a similar pattern to the `wasm3-arduino` project, whereby the file is converted
to a header file containing the bytes directly encoded inline:

<FileHeader>app.wasm.h</FileHeader>

```cpp
unsigned char build_optimized_wasm[] = {
  0x00, 0x61, 0x73, 0x6d, 0x01, 0x00, 0x00, 0x00, 0x01, 0x3a, 0x0a, 0x60,
  ...
};
unsigned int build_optimized_wasm_len = 4409;

```

From here, loading the WASM module is simple:

<FileHeader>main.cpp</FileHeader>

```cpp
int main(void) {
    IM3Environment env = m3_NewEnvironment ();
    IM3Runtime runtime = m3_NewRuntime (env, 1024, NULL);
    IM3Module module;
    m3_ParseModule (env, &module, build_optimized_wasm, build_optimized_wasm_len);
    m3_LoadModule (runtime, module);
    LinkNDSFunctions(module);
    IM3Function f;
    m3_FindFunction (&f, runtime, "start");
    m3_CallV (f, 10);
    return 0;
}
```

This is the simplest possible main function to load a WASM module, with any kind of error checking omitted. An important part here is `LinkNDSFunctions`
from `nds.cpp`, which includes all the functions that should be accessible from the WebAssembly interface. For example for the basic print function, it's possible
to create a WebAssembly accessible function using the `m3ApiRawFunction` macro:

<FileHeader>nds.cpp</FileHeader>

```cpp
m3ApiRawFunction(_print)
{
    m3ApiGetArgMem (const char *, str)
    iprintf(str);
    m3ApiSuccess();
}
```

Here `m3ApiGetArgMem` is setting up the pointer to point to the correct location along the WASM linear memory.
The new `_print` function is then exposed to the WASM module like so:

<FileHeader>nds.cpp</FileHeader>

```cpp
    m3_LinkRawFunction (module, "nds", "_print", "v(i)", &_print);
```

Here `v(i)` denotes the signature of the function - in particular, this function takes an integer parameter (a pointer to a string) and returns void.

On the AssemblyScript side of things, an important thing to note is that when passing strings to the exposed C++ functions, it's necessary to byte-encode
them and provide a null terminator:

<FileHeader logo="assemblyscript">nds.ts</FileHeader>

```typescript
const encode = (str: string): ArrayBuffer => String.UTF8.encode(str, true);

declare function _print(ptr: ArrayBuffer): void;
export function print(str: string): void {
  _print(encode(str));
}
```

When the exposed C++ functions are called from within the WASM environment, a pointer to the byte-encoded string is passed,
and the C++ functions can read the sequence of characters directly from the WASM linear memory.

The result is a general-purpose C++ `main.cpp` entry point and a boilerplate linking layer `nds.cpp` exposing the native APIs to the WASM environment.

The WASM module containing the game logic can be replaced by any other module and access the same exposed native API calls without any changes to the native compiled C++ code.

## Building the game

As with any project with a relatively complex set of dependencies, I used (and highly recommend) VSCode's [container remote](https://code.visualstudio.com/docs/remote/containers) feature.

For this project, I used the [devkitarm docker image](https://hub.docker.com/r/devkitpro/devkitarm), a pre-configured environment for easily cross-compiling
NDS roms.

The AssemblyScript is compiled to WASM, which is in turn encoded as a byte array in `app.wasm.h`, and then this is compiled directly into the
final NDS rom:

<figure>
  <img src="/blog/2021/09/snake-ds-build.svg" />
</figure>

While this method involves embedding WASM in the rom itself, there's no particular requirement for this to be the loading mechanism.
Since the WASM is interpreted at runtime without any AOT compilation, the WASM could also be fetched dynamically over a network or read from disk and executed.

## The End Result

The compiled rom can be run on either a DS emulator or a
physical device. Here is the game running in [DeSmuME](https://desmume.org/), a Nintendo DS emulator:

<figure>
  <img src="/blog/2021/09/snake-assemblyscript-ds.gif" />
</figure>

The code for the game, as well as a build pipeline and compiled releases can be found here:

<Github
  author="moxon6"
  repo="snake-assemblyscript-ds"
  description="Snake DS written in AssemblyScript"
/>

## Attributions

- _Image: [Nintendo DS Lite](https://commons.wikimedia.org/wiki/File:Nintendo-DS-Lite-Black-Open.jpg) (Modified): [Evan-Amos](https://commons.wikimedia.org/wiki/File:Nintendo-DS-Lite-Black-Open.jpg) [CC BY-SA 3.0](https://creativecommons.org/licenses/by-sa/3.0) , via Wikimedia Commons_


One of the most common misconceptions about WebAssembly comes from the name itself.

WebAssembly On Your Nintendo DS


<figure>
  <img src="/blog/2021/06/binary-wasm.svg" />
</figure>

A project I'm working on led me to attempt to create a reusable package from an existing WebAssembly tool by utilising modern tooling and APIs.

The tool is [quiet.js](https://quiet.github.io/quiet-js/) - a WebAssembly build of [libquiet](https://github.com/quiet/quiet) for encoding and decoding binary data to and from audio.

The main motivation in this case for compiling this project to WebAssembly is that the end user effectively gains the benefits of a compiled executable (in particular not having to download and compile multiple dependencies) whilst also being available on any modern browser and on any architecture.

Since I wanted to use the tool in a project of my own, there were a few shortcomings that I felt could be addressed:

- quiet.js is not an npm package, nor structured in such a way that it could be easily be used as one, making it difficult to use as part of an application.
- It is compiled using Emscripten and dependent on a large amount of generated Emscripten JavaScript glue code, and only accessible via via the global `Module` object. 
  Modern browser APIs make it relatively easy to interface directly with WebAssembly modules and access their exports directly.
- The quiet.js Transmitter and Receiver logic is tightly coupled, and the control flow somewhat complex.
- quiet.js depends on some deprecated Web Audio APIs, such as `ScriptProcessorNode`.
- The build process for the WASM module itself was not available, making it difficult to determine exactly what source went into the compiled WebAssembly.

I decided to attempt a rewrite of quiet.js with a goal of resolving these issues.

## Standardising the build process

While the existing WASM is available as a pre-compiled object in the repository source, I felt an important first step was to reproduce the build process. 

<figure>
  <img height="200px" src="/blog/2021/06/compile-wasm-2.svg" />
</figure>

By building the WASM in in an openly accessible build pipeline, contributors can collaborate not only on the JavaScript interface code but also on the compilation steps of the WASM.

Additionally, providing a start to finish build script enables the project to 
adhere to the LGPL licensing of some of the dependencies, since the statically 
linked code is accompanied with tools that could be used to create a build
using different versions of the LGPL components.

Building the WASM version was a little tricky, since many of the compilation steps assume the compilation target is native executable code, 
and that dependencies will exist in certain locations on disk.

While this was easier to resolve for most of 
the libraries through use of various compiler flags, the current pipeline definition does currently 
include a very specific `CMakeLists.txt` patch that is tightly coupled to the version of libquiet it 
is building.

The full pipeline for building the WASM [can be found here](https://github.com/moxon6/quiet-js/blob/master/.github/workflows/build-wasm.yml)

## Replacing the Emscripten Glue Code

The accompanying glue code generated by Emscripten amounts to over 650kb of minified JavaScript. By targeting the specific functions needed from libquiet, 
it was relatively simple to replace the functionality provided by Emscripten, as well as having more fine-grained control over the compilation and instantiation of the WASM.

<figure>
  <img height="120px" src="/blog/2021/06/replace-emscripten.svg" />
</figure>

The first step involves fetching and instantiating the WASM using the streaming compilation API.

```javascript
const { instance } = WebAssembly.InstantiateStreaming(fetch('quiet.wasm'))
```

A major benefit of decoupling the WASM build step from JavaScript specific Emscripten features is that the 
WASM output can potentially run on other WebAssembly runtimes, such as [Wasmer](https://wasmer.io/).

Additionally, loading the WASM this way instead of via the existing Emscripten glue code
avoids polluting the global namespace with the `Module` object (there are also Emscripten build flags 
than can generate an ES module version of the JS glue code).

One part that required replacing was logic for handling non-numeric data types, in particular 
strings. 
Whilst WebAssembly currently only supports numeric values as parameters,
it is possible to pass strings between JavaScript and WebAssembly 
by reading and writing bytes directly to and from the WebAssembly linear memory.


<figure>
  <img height="200" src="/blog/2021/06/linear-memory.svg" />
  <figcaption>"Softwayre" in WASM linear memory</figcaption>
</figure>

<FileHeader>utils.js</FileHeader>

```javascript
...

function allocateArrayOnStack(instance, arr) {
  const ret = instance.exports.stackAlloc(arr.length);
  const HEAP8 = new Int8Array(instance.exports.memory.buffer);
  HEAP8.set(arr, ret);
  return ret;
}

...
```

Similarly, managing things like the stack pointer (which is typically managed for you when compiling C) must be explicitly handled by calling `stackSave()` and `stackRestore()` at appropriate points:

```javascript
const stack = instance.exports.stackSave();

// Operations that allocate to stack here

instance.exports.stackRestore(stack)
```

### Caveat: WASM imports
Since some of the dependencies of libquiet depend on system APIs such as the file system, these are automatically specified in the compiled WASM as imports. 

<FileHeader>quiet.wat</FileHeader>

```wasm
(import "wasi_snapshot_preview1" "proc_exit" (func (;0;) (type 0)))
(import "wasi_snapshot_preview1" "clock_time_get" (func (;1;) (type 65)))
(import "wasi_snapshot_preview1" "fd_close" (func (;2;) (type 1)))
(import "wasi_snapshot_preview1" "fd_write" (func (;3;) (type 8)))
(import "wasi_snapshot_preview1" "fd_seek" (func (;4;) (type 66)))
(import "wasi_snapshot_preview1" "fd_read" (func (;5;) (type 8)))
(import "env" "__sys_getpid" (func (;6;) (type 7)))
```
Normally it would make sense to use a WASI implementation to provide these imports, such as `@wasmer/wasi`:
 
<NPMPackage name="@wasmer/wasi" author="Wasmer Engineering Team" />

which includes implementations for the required imports above, such as a virtual file system.

However, since the parts of the library this tool exposes definitely won't be using any of the functions that access the system APIs,
I simply provided stub no-op functions as imports. 

__This is a workaround__: A later step may be to optimise the build process to not depend on any file system APIs and therefore not require these module imports. 

## Modernising Web Audio API usage

The quiet.js core code makes heavy use of the Web Audio API to send Audio data to the user's speakers and receive data from the user's microphone.

### Re-implementing the Transmitter

<figure>
  <img height="200px" src="/blog/2021/06/transmitter.svg" />
</figure>

An issue with the original quiet.js Transmitter implementation is that it used [`ScriptProcessorNode`](https://developer.mozilla.org/en-US/docs/Web/API/ScriptProcessorNode) to 
both to transmit audio as well as process incoming audio from the Microphone. 

This is now a deprecated API, and for the case of playback of data in memory, [`AudioBufferSourceNode`](https://developer.mozilla.org/en-US/docs/Web/API/AudioBufferSourceNode) is a more modern approach.

Re-implementing the Transmitter was relatively simple - playing the audio from the speakers was a matter of creating an `AudioBufferSourceNode` from the encoded data in the audiobuffer and connecting it to the existing audioContext.

<FileHeader>Transmitter.js</FileHeader>

```javascript
...
  function transmit() {
      ...
      const audioBufferNode = new AudioBufferSourceNode(audioContext);
      audioBufferNode.buffer = audioBuffer;
      audioBufferNode.connect(audioContext.destination);
      audioBufferNode.start(t);
      t += audioBuffer.duration;
      ...
  }
...
```

Here the `t` is the playback time offset, which is necessary for when the input exceeds the size of a single buffer. By keeping a running duration total each frame is transmitted sequentially.

### Re-implementing the Receiver

<figure>
  <img height="200px" src="/blog/2021/06/receiver.svg" />
</figure>

The original Receiver logic also made use of `ScriptProcessorNode` to process incoming audio from the Microphone. 

This is now deprecated in favour of AudioWorklets, which can process data in a separate AudioWorklet thread.

AudioWorklets provide several benefits, including not running on the [main thread](https://developer.mozilla.org/en-US/docs/Glossary/Main_thread), as well as providing stronger guarantees about the real-time nature of the incoming data.

However, switching to use AudioWorklets posed two additional problems:

The first is that The AudioWorkletThread cannot make network requests, and getting access to the WASM on the AudioWorkletThread meant taking a slightly different approach to the one taken for Transmitting encoded data.

Having read a similar problem [discussed in the google developer blog](https://developers.google.com/web/updates/2018/06/audio-worklet-design-pattern), as well as the [MDN documentation](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/WebAssembly/Module) for `WebAssembly.Module`, I found that unlike a `WebAssembly.Instance`, an uninstantiated `WebAssembly.Module` can be serialised and passed between threads.

<figure>
  <img height="125px" src="/blog/2021/06/wasm-threading.svg" />
  <figcaption>Passing the WASM module from the main thread to the Web Audio thread</figcaption>
</figure>

The WebAssembly compilation step was updated as so:

```javascript
const { instance, module } = WebAssembly.InstantiateStreaming(fetch('quiet.wasm'))
```

While the instance object is used on the main thread, the module is passed to the AudioWorkletThread
which can then instantiate it into an instance of its own and use it in the process method of the custom AudioWorklet:

<FileHeader>index.js</FileHeader>

```javascript

this.quietProcessorNode = new AudioWorkletNode(
  this.audioContext, 
  'quiet-receiver-worklet', 
  {
    processorOptions: {
      quietModule: module
    },
  }
);
```

<FileHeader>quiet.audioworklet.js</FileHeader>

```javascript

class ReceiverWorklet extends AudioWorkletProcessor {
  constructor(options) {
    super();
    const { quietModule } = options.processorOptions;
    ...
  }
}
```

The second problem with transitioning to AudioWorklets is that unlike ScriptProcessorNode which could provide the required number of bytes on demand, `process()` would give 128 bytes at a time, and decoding the data was not initially possible. 

Fortunately, the blog post mentioned above also described a pattern for using a [RingBuffer](https://github.com/GoogleChromeLabs/web-audio-samples/blob/656f7cf2241d316b4c87b6e65275ff1684724b1a/audio-worklet/design-pattern/lib/wasm-audio-helper.js) to buffer an arbitrary number of bytes.


Using this pattern, the custom `AudioWorkletNode` can decode incoming data and use `postMessage()` to send the decoded string back to the main thread.

## Packaging it all up for npm

An added layer of complexity in packaging this into an npm package is that there are two files that need special consideration - the `quiet.wasm`, 
which should be fetched asynchronously at runtime, and the quiet AudioWorklet, which as well as being in a separate thread is also in it's own file.

Using an AudioWorklet involves fetching the WASM at runtime:

```javascript
await WebAssembly.instantiateStreaming(fetch(PATH_TO_WASM))
```

and similarly evaluating the path to the AudioWorklet file at runtime:

```javascript
audioContext.audioWorklet.addModule(PATH_TO_MODULE_JS)
```

When trying to use the package with other bundlers such as [Parcel](https://parceljs.org/) and [Webpack V4](https://v4.webpack.js.org/), 
the dependencies were not detected and bundled, and were not accessible from the compiled package.

There are several ways these issues can be resolved, though they typically require either bundling the external assets as Blobs in the main entry file
(which has it's own performance downsides) or using non-standard syntax, resulting in code that may be tied to a specific bundler implementation.

Fortunately, [Webpack V5](https://webpack.js.org/blog/2020-10-10-webpack-5-release/) implements several ECMAScript compatible features for
handling these cases - in particular [Asset Modules](https://webpack.js.org/guides/asset-modules/) and [Web Worker Support](https://webpack.js.org/guides/web-workers/)

Webpack V5 can detect relative paths to assets in dependencies by using `URL` and `import.meta.url`, like so for the WASM:

<FileHeader>index.js</FileHeader>

```javascript
 await WebAssembly.instantiateStreaming(fetch(new URL('./quiet.wasm', import.meta.url)))
```

and for the AudioWorklet:

<FileHeader>index.js</FileHeader>

```javascript
audioContext.audioWorklet.addModule(new URL('./quiet.worklet.js', import.meta.url))
```

Using Webpack as the bundler for application code allows the package to be written in such a way that it is compatible with native 
[`import.meta`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import.meta) 
ES module syntax, which is already implemented in most modern browsers.

However, this did not solve every issue - when the modules are bundled together and transpiled, an implementation assumption is that all bundles have access to the same global `require` scope.
This is false, however, for the AudioWorklet, which runs on the AudioWorkletGlobalScope and so any `import` statements would fail to resolve.

In order to resolve this issue, I used rollup as a build step for the package to resolve imports _before_ publishing, producing a single ES module for both the main bundle as well as the AudioWorklet.

The result is @moxon6/quiet-js:

<NPMPackage 
  name= "@moxon6/quiet-js"
  author="Martin Moxon" />

A live demo is [available here](https://moxon6.github.io/quiet-js), as well as an editable CodeSandbox below:

<CodeSandbox sandboxId="moxon6quiet-js-g7ep8" />



A project I'm working on led me to attempt to create a reusable package from an existing WebAssembly tool by utilising modern tooling and APIs.

Blog

Softwayre Blog Posts - Martin Moxon

WebAssembly On Your Nintendo DS

Packaging a WebAssembly Audio Tool

Browse by tech

Browse by author